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ABSTRACT 

Let  P=P,,...,Pm  and  n=Q  . .  ,,Q  be  two  patterns  of 
points.  EaCh  pairing  (P.,Qh  of  a  point  of  P  with  a 
point  of  0  defines  a  relative  displacement  «S  i  j  of  the 
two  patterns.  We  can  define  a  figure  of  merit  for 
according  to  how  closely  other  point  pairs  coincide 
under  iS^j.  If  there  exists  a  displacement  60  for  which 
P  and  match  reasonably  well,  the  pairings  for  which 
Aii=Ao  will  have  high  merit  scores,  while  other  pairings 
will  not.  The  scores  can  then  be  recomputed,  giving 
weights  to  the  other  point  pairs  based  on  their  own 
scores;  and  this  process  can  be  iterated.  When  this  is 
done,  the  scores  of  pairs  that  correspond  under  <SG  remain 
relatively  high,  while  those  of  other  pairs  become  low. 
Examples  of  this  method  of  point  pattern  matching  are 
given,  and  its  possible  advantages  relative  to  other  methods 
are  discussed. 
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1.  Introduction 


In  [1]  some  experiments  in  point  pattern  matching  are 
described.  Given  two  point  patterns  P=P^ ,  .  .  .  , P^  and 
Q^#...#Qn#  we  count,  for  each  displacement  ^  of  P  relative 
to  (J,  how  many  pairs  (P^,Qj)  lie  closer  together  than  some 
threshold  t.  If  P  and  Q.  have  many  points  in  common,  this 
process  will  yield  a  large  match  peak  for  that  6  that  maps 
these  points  into  themselves,  while  others  ^'s  will  yield 
at  best  low  match  values  where  a  few  pairs  happen  to  coincide. 

The  matching  process  just  described  is  essentially  cauiv- 
alent  to  cross-correlating  P  with  a  "blurred"  version  of  Q. 
in  which  each  point  has  been  expanded  into  a  disk  of  radius 
t.  (Actually,  for  computational  simplicity,  only  displace¬ 
ments  that  map  some  P^tP  into  some  wore  considered.) 

It  will  tolerate  local  distortions,  as  long  as  they  do  not 
generally  give  rise  to  relative  displacements  that  exceed  t. 

On  the  other  hand,  t  must  be  substantially  smaller  than  the 
average  interpoint  distance  in  P  and  C,  since  otherwise  many 
false  matches  will  be  detected. 

This  paper  discusses  an  alternative  approach  to  point 
pattern  matching  in  which  a  figure  of  merit  is  assigned  to 
each  pair  (P^,Qj),  according  to  how  closely  other  pairs 
(Ph»Qk)  match  when  P^  is  mapped  into  Oj  •  Let  be  the 

displacement  that  maps  P^  into  ;  if  there  exists  a  dis¬ 
placement  6^  for  which  P  and  2  match  reasonably  well,  and 


6^  is  close  to  6^,  then  (P^,Q^)  will  have  a  high  merit 
score,  but  otherwise  it  will  not.  The  scores  can  then  be 
recomputed,  giving  weights  to  the  other  point  pairs  (P^Q^) 
based  on  their  own  scores;  and  this  process  can  be  iterated. 
When  this  is  done,  the  scores  of  pairs  that  correspond  under 
6^  remain  relatively  high,  while  those  of  other  pairs  become 
low. 

The  merit  score  computation  is  described  in  greater 
detail  in  Section  2,  and  some  examples  of  point  pattern 
matching  by  this  method  are  given  in  Section  3.  For  earlier 
work  on  the  use  of  iterative  ("relaxation")  methods  in  image 
matching  see  12]. 


2.  Match  merit  computation 


When  we  pair  with  Q  ^ ,  we  would  like  to  find  Q's  in 

the  same  positions  relative  to  Qj  that  the  P's  have  relative 

to  P^.  Let  Ph  and  Qk  be  any  P  and  Q  other  than  Pi  and  P_.  , 

and  let  6^ (h,k)  be  the  position  difference  of  P^  and 

when  P.  is  mapped  into  Q..  If  |6_..(h,k)|  is  zero,  Q,  is 

exactly  in  the  same  position  relative  to  that  P^  is 

relative  to  P^,  so  that  the  pair  (P^Q^)  should  give  the 

pair  (P^Qj)  maximal  support;  while  as  |6^j(h,k)|  increases, 

this  support  should  decline.  Let  the  support  given  to 

(P.  Q-)  by  (P  ,Q,  )  be  denoted  by  cp ( 1 6.  .  (h,k)  |).  In  the  ex- 
i »  j  n  x  13 

periments  described  in  the  next  section,  we  used  ; 

many  other  functions  could  have  been  used  instead. 

For  each  P^,  there  may  be  several  Q^’s  that  lie  close 
to  it;  but  we  only  want  one  Q,  to  correspond  to  P^  when  P. 
is  paired  with  Q^.  Thus  it  is  reasonable  to  define  the  sup¬ 
port  to  associated  with  P^  as 


k?j[cp(|6ij(h'k)  1)1  (1) 

In  this  way,  we  can  compute  the  support  provided  to  (P^,Qj) 
by  each  Ph»  To  obtain  the  total  support  for  (P^Qj),  we 
average  the  contributions  of  all  the  P^'s: 


In  computing  s 


we  treated  all  pairs  (Ph,Qk)  equally, 
since  a  priori  any  two  points  can  correspond.  Now,  however, 
we  can  recompute  the  support,  taking  into  account  the  fact 
that  each  pair  has  a  figure  of  merit  s^  •  Specifi¬ 

cally,  the  support  given  to  (P^ ,Q ^ )  by  (Ph*Qk>  should  depend 
not  only  on  the  position  difference  between  Ph  and  Q^,  but 
also  on  their  s^  value.  These  two  factors  can  be  combined 
in  various  ways;  we  have  chosen  to  use  their  minimum,  i.e., 
to  define  the  support  for  (P.,Q.)  given  by  (Ph,Qv)  as 

min[«p(  |6ij  (h,k)  |  )  ,  s(0)(Ph,Qk)]  (3) 

We  can  then  compute  the  support  for  (P^,Qj)  associated  with 
Ph  as 

™“(min(^(|«i:j(h,k)  |),  s(0)  (Ph,Qfc)  ]  )  (1') 

and  the  total  support  for  (P^,Qj)  from  all  the  P^'s  as 

s<1’  ‘Wtct  1 6ij (h'k>  I » •s<0)  <ph-°k> 1  > J 

(2*) 


This  process  can  then  be  iterated;  at  the  rth  step  we  have 

s<r>  (pi'Qj)=5=T  hJifk^j(minI‘t'(|,5ij(h'k»  D  's<r"1)<ph'0k)n 


3.  Experiments 


The  process  described  in  Section  2  was  applied  to  the 
same  data  sets  used  in  11].  In  each  of  the  following  figures, 
parts  (a-b)  show  the  two  point  patterns  P  and  Q.  Part  (c) 
tabulates ,  for  each  P^tP,  the  two  points  Q^tO  for  which  the 
match  merit  of  the  pair  (P^Q^)  is  highest.  Part  (d)  shows 
the  total  match  merit  associated  with  each  relative  displace¬ 
ment  of  P  and  0,  multiplied  by  100. 

The  point  patterns  in  Figures  1  and  2  represent  two  sets 
of  local  features  extracted  from  a  picture  of  a  tank  and  from 
a  road  map  of  the  Washington,  D.C.  area,  respectively.  In 
each  case,  there  is  a  strong  match  peak  corresponding  to  the 
proper  relative  displacement  of  the  two  patterns.  Note  that 
the  merit  of  the  second  best  is  not  always  very  much  lower 
than  that  of  the  best  0j,  but  it  is  often  at  least  twice  as 
low. 

Figures  3-5  show  results  for  sets  of  edge  points  extracted 
from  a  succession  of  FLIR  images  of  a  tank.  The  comparisons 
are  for  frames  1  and  10,  1  and  5,  and  5  and  10,  respectively'. 
The  smearing  of  the  peaks  indicates  the  motion  of  the  tank 
from  frame  to  frame.  Figures  6-7  show  results  using  two  dif¬ 
ferent  edge  detectors  on  an  image  of  a  tank  and  of  an  APC. 

Figures  8-11  show  the  results  of  applying  various  amounts 
of  "random  walk  noise"  to  the  second  set  of  map  feature  points 


used  in  Figure  2.  The  random  jumps  had  uniformly  distributed 

directions;  their  magnitudes  were  normally  distributed  with 

mean  0  and  standard  deviation  3%,  4^%,  6%,  and  9%  (of  the 

picture  diameter)  in  Figures  8-11,  respectively.  Good  match 

peaks  are  obtained  for  the  first  three  cases. 

Figures  12-13  show  the  results  of  rotating  one  of  the  map 

feature  point  sets  by  5°  and  10°  relative  to  the  other.  Good 

match  peaks  are  obtained  in  both  cases;  in  (1),  on  the  other 

hand,  the  10°  rotation  did  not  yield  a  good  peak. 

In  all  of  these  examples,  the  results  shown  are  for  the 

( 4 ) 

fourth  iteration  (i.e.,  s  (P^,0j)).  Figure  14  shows  the 

results  of  iterations  0,1, 2, 3  for  the  case  in  Figure  1.  Note 

that  even  for  s^,  the  peak  is  four  times  higher  than  the 

largest  non-peak  values,  and  that  this  ratio  is  (at  least) 

maintained  while  the  number  of  non-peak  values  drops  substanti- 

( 3 ) 

ally.  Note  also  that  in  this  case  the  s  values  are  the  same 
( 4 ) 

as  the  s  values  in  Figure  ld--i.e.,  the  process  has  stabil¬ 


ized  after  three  iterations. 


4. 


Discussion 


The  relaxation  scheme  for  point  pattern  matching 
described  in  this  paper  is  comparable  in  computational 
cost  to  the  simpler  scheme  of  11] .  Both  schemes  consider 
only  the  mn  displacements  that  pair  off  all  possible  points 
of  P  with  points  of  <2;  and  for  each  such  displacement,  they 

compare  all  other  points  of  P  with  all  other  points  of  0. 

2  2 

Thus  the  total  amount  of  computation  is  (Mm  n  ).  The  indi¬ 
vidual  computations  involved  in  the  relaxation  scheme  are 
somewhat  more  complex,  and  must  be  iterated  several  times; 
but  this  additional  complexity  represents  only  a  constant 
factor  which  does  not  grow  with  m  and  n. 

On  the  other  hand,  the  relaxation  approach  appears  to  be 
more  tolerant  to  global  distortion  than  the  method  of  |1]. 

For  example,  consider  the  case  where  one  pattern  is  rotated 
relative  to  the  other.  This  rotation  shifts  different  parts 
of  the  pattern  in  different  directions;  hence  when  we  compute 
a  match  score  for  any  given  relative  shift,  as  in  [1],  it  will 
be  relatively  low.  On  the  other  hand,  for  any  given  point  P, 
there  will  be  neighboring  points  that  are  shifted  in  approxi¬ 
mately  the  same  way  as  P,  so  that  in  the  relaxation  scheme 
these  points  will  provide  some  support  for  pairing  P  with  an 
appropriate  Q.  Thus  the  relaxation  method  should  give  better 
results  under  rotation  than  the  method  of  (1),  and  this  is 


indeed  the  case  (cf.  Figures  12-13). 

In  summary,  the  relaxation  approach  deserves  consideration 
as  a  method  of  evaluating  point  pattern  matches  under  distortion. 
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Figure  2d 


lpoint  match  confidence  next  best 


h' 

fv 

fv 

CM 

rv 

n 

CD 

N 

4- 

■o 

•0 

o 

in 

O' 

o 

00 

m 

CM 

(M 

(M 

CM 

^4 

•H 

O 

o 

CM 

CM 

CM 

ru 

o 

*H 

o 

o 

O 

O 

O 

O 

O 

o 

O 

o 

o 

O 

O 

O 

o 

o 

O 

o 

o 

^4 

^4 

v4 

^4 

H 

^4 

-0 

>0 

4) 

•o 

O' 

O' 

rM 

T-< 

•H 

^4 

^4 

^4 

H 

w4 

rM 

t-C 

Cd 

n 

4 

<* 

4 

CM 

CM 

CM 

CM 

»*4 

CM 

n 

4 

O' 

O' 

lO 

in 

^4 

rH 

««4 

H 

H 

♦4 

H 

r4 

H 

H 

H 

rM 

H 

H 

-0  O'  O'  O' 

O' 

O' 

■o 

CM 

in 

>0 

'O 

O' 

tv 

|v 

N 

N 

in 

m 

-o 

in 

CM  cm  cm  cm 

CM 

CM 

CM 

CM 

CM 

CM 

CM 

H 

CM 

CM 

CM 

CM 

CM 

CM 

CU 

CM 

oooo 

o 

O 

O 

O 

o 

o 

o 

o 

O 

O 

O 

O 

o 

o 

o 

o 

*■4  rM 

w—4  rH  rH  »H 

r-4 

T-C 

•■4 

v4 

«-4 

^4 

O' 

O' 

O' 

O' 

>0 

>0 

>0 

o 

N 

N 

O' 

O' 

CM  CM  CM  n 

»H  rM  H 

xT 

m»4 

in 

m 

in 

*4 

O' 

O' 

O' 

O' 

12 

13 

*■4 

15 

|v 

H 

IV 

19 

O' 

OOOO 

^  H  H  H 

O 

^4 

10 

O 

w-4 

o 

M-4 

O' 

O' 

IV 

4) 

in 

m 

in 

m 

in 

m 

CD 

N 

n«tmo 

IV 

m 

O' 

o 

H 

CM 

H 

^4 

in 

>0 

N 

OD 

o 

o 

CM 

CM 

^  r< 

♦■4 

*■4 

H 

CM 

*■4 

r4 

H 

^4 

T-4 

•H 

H 

*-4 

H 

CM 

CM 

CM 

4fflUQ 

W 

U. 

o 

I 

»-4 

O 

_J 

r 

Z 

O 

CL 

O 

a: 

cn 

»- 

o 


lou.  WQucoc 

. -><-> 

w . 

H . SC 

. _l 

eoilozl 


.O 


Figure  3  (a-c) 
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Figure  14a 


Figure  14b 
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